1. Field of the Invention
The present invention relates to digital signal processing, and in particular to a media processing apparatus that decompresses compressed video data and compressed audio data, and can also compress video data and audio data and perform graphics processing.
2. Description of the Related Art
With the development of compressing/decompressing techniques for digital video data and the improvements in LSI (Large Scale Integration) technology in recent years, various media processing circuits have come to be highly valued. Examples of such are a decoder that decodes (decompresses) compressed video and audio data, an encoder that compresses video and audio data, and a graphics processor that processes graphics.
As one example of conventional techniques, an AV (Audio Visual) decoder decodes video data and audio data compressed according to MPEG (Moving Picture Experts Group) standard. This AV decoder uses two processors, with one processor decoding video data and the other processor decoding audio data.
FIG. 1 shows a representation of the decode processing performed by this AV decoder. In FIG. 1, the vertical axis represents time and the horizontal axis represents the respective computational complexity of the video decoding performed by the video processor and the audio decoding performed by the audio processor.
The video processor alternately performs sequential processing for compressed video data in macroblock units and decode processing for the actual video data. The sequential processing part of the video decoding analyzes the header part of a macroblock being processed, and so is hereinafter referred to as xe2x80x9cheader analysisxe2x80x9d. The decode processing decodes the compressed video data in a macroblock, and so is hereinafter referred to as xe2x80x9cblock decodingxe2x80x9d. Of these, header analysis requires judgement of various conditions and has a low computational complexity. Block decoding decodes the variable-length codes in an MPEG stream using the various kinds of data produced by the header analysis, and also performs IQ (Inverse Quantization) and IDCT (Inverse Discrete Cosine Transform) processing in block units. This means that the block decoding has a high computational complexity.
The audio processor alternately performs sequential processing for compressed audio data and decoding processing for the actual audio data.
The above technique has the following problems.
The first problem is the cost of manufacturing this AV decoder. In particular, the video processor needs to be a high performance processor that can perform real-time processing on a large amount of video data. This means that a processor with a high clock speed must be used. Such processors are expensive.
When the media processing apparatus described above is used in an AV decoder provided in a digital (satellite) broadcast tuner (called an STB (Set Top Box)) or a DVD (Digital Versatile/Video Disc) player, MPEG streams received on a broadcast wave or read from a disc are inputted. The AV decoder decodes these MPEG streams, and outputs video signals and audio signals to a display and speakers, respectively. This series of processes requires a huge amount of processing. For this reason, there has been increasing demand for a media processing apparatus which can efficiently execute this huge amount of processing.
It is a first object of the present invention to provide a media processing apparatus that performs a series of processes composed of the input of stream data that expresses compressed video and compressed audio, decoding of the data, and output of the result, the media processing apparatus having a high performance without operating at a high clock frequency. This reduces the manufacturing cost of the media processing apparatus.
The first object can be achieved by a media processing apparatus that decodes a data stream including compressed audio data and compressed video data to obtain video data and audio data, the media processing apparatus including: a sequential processing unit for performing a header analysis process on the compressed video data and an audio decode process on the compressed audio data, the header analysis process being given priority over the audio decode process and being a header analysis for a predetermined block included in the compressed video data in the data stream, and the audio decode process being a decode process for the compressed audio data; and a routine processing unit for performing a decode process for the compressed video data aside from the header analysis process, the routine processing unit operating in parallel with the sequential processing unit.
With the stated construction, the processing load of the decoding of compressed video data is shared by the sequential processing unit and the routine processing unit, so that the processing load is less than when the entire decode was conventionally handled by a single processor. This raises the overall processing efficiency. The sequential processing unit performs the header analysis process with priority over the audio decode process, so that the decode process of the routine processing unit that is based on the header analysis is performed without delay. This speeds up the overall processing of the media processing apparatus. The audio decode process requires considerably less processing than the decode process for the compressed video data, and is performed by the sequential processing unit in between header analysis processes which also results in higher processing efficiency. Consequently, the media processing apparatus of the present invention achieves a high performance without operating at a high clock frequency, thereby reducing the manufacturing cost of the media processing apparatus.
Here, the sequential processing unit may include a processor and a memory that stores a header analysis thread, an audio decode thread, and a thread switching thread, the processor executing the threads in the memory to realize: a header analysis unit that performs the header analysis process as a result of the processor executing the header analysis thread in the memory; an audio decode unit that performs the audio decode process as a result of the processor executing the audio decode thread in the memory; and a control unit that controls a switching of a thread assigned to the processor so that the header analysis process is given priority over the audio decode process, the control unit being realized by the processor executing the control thread.
With the present construction, the control unit assigns threads to the processor so that the header analysis process is executed with priority. As a result, the media processing apparatus smoothly switches between the header analysis process and the audio decode process and so can process data streams with high efficiency.
Here, the control unit may include: an interrupt detection unit for detecting an occurrence of any of a plurality of interrupts that cause a switching of thread; a state management unit for managing a state of the header analysis unit and a state of the audio decode unit based on a type of detected interrupt, each state being one of an execute state, a wait state, and a ready state; and a switching unit for switching, when the state of the header analysis unit has changed from the wait state to the ready state, a thread assigned to the processor to the header analysis thread.
With the stated construction, the control unit has an interrupt detection unit, a state management unit, and a switching unit. The interrupt detection unit detects interrupts. The state management unit manages changes in the states of the header analysis process and the audio decode process that are caused by interrupts detected by the interrupt detection unit. The switching unit assigns the header analysis thread to the processor when the state of the header analysis unit changes to the ready state. As a result, the control unit is a system that is activated by interrupts, so that there is no need to constantly monitor which of the header analysis unit and the audio decode process should be set in the execute state.
Here, the state management unit may include: a state table for storing the state of the header analysis unit and the state of the audio decode unit; and a state updating unit for updating the state table based on the type of detected interrupt.
Here, the predetermined block may be a macroblock included in the compressed video data in the data stream, and the interrupt detecting unit may be capable of detecting a type 1 interrupt and a type 2 interrupt, the type 1 interrupt being outputted by the header analysis unit on completing the header analysis process for one macroblock and the type 2 interrupt being outputted by the routine processing unit on completing the decode process for one macroblock, the state updating unit changing the state of the header analysis unit in the state table to the wait state when the interrupt detecting unit has detected the type 1 interrupt and changing the state of the state of the header analysis unit in the state table to the ready state when the interrupt detecting unit has detected the type 2 interrupt.
With the stated construction, when the interrupt detection unit detects the type 1 or type 2 interrupt, the state updating unit updates the state of the header analysis unit in the state table to the wait state or the ready state. When the state of the header analysis process has changed to the ready state, the switching unit assigns the header analysis thread to the processor. This means that when the decoding of the header part has been completed by the routine processing unit, the header analysis process will soon be commenced. This improves the processing efficiency.
Here, when the header analysis unit is in the wait state and the state of the audio decode unit has changed from the wait state to the ready state, the switching unit may switch a thread assigned to the processor to the audio decode thread.
With the stated construction, when the state of the header analysis unit becomes the ready state, the switching unit assigns the header analysis thread to the processor even if the audio decode unit is presently in the execute state. This means that the header analysis process is executed with priority over the audio decode process, which raises the overall processing efficiency of the media processing apparatus.
Here, the routine processing unit may include: a data conversion unit for performing a variable length decode on the compressed video data; a calculation unit for performing an inverse quantization and an inverse discrete cosine transform on image blocks resulting from the variable length decode; and a synthesizing unit for performing a motion compensation process by synthesizing image blocks resulting from the inverse discrete cosine transform with previously decoded image blocks to produce decoded image data, and the header analysis unit may include: an obtaining unit for issuing obtaining header information that has been subjected to a variable length decode by issuing an indication to the data conversion unit; an analysis unit for performing analysis on the obtained header information and then calculating motion vectors; a notification unit for notifying the routine processing unit of parameters obtained by the analysis of the analysis unit; and a start indication unit for issuing a start indication to the data conversion unit at a predetermined point during the header analysis processing, the start indication having the data conversion unit start a variable length decode on compressed image data aside from the header information.
With the stated construction, the decode processing of compressed video data is split between the analysis unit and the routine processing unit. In the routine processing unit, processing centered on a routine calculation of data is executed by a pipeline architecture including the data conversion unit, the calculation unit, and the synthesizing-unit. In the header analysis unit, the analysis unit performs the header analysis that is processing which accompanies a judgement of conditions, and the notification unit notifies the routine processing unit of the parameters obtained as the analysis result. This means that the decode process for the compressed video data is divided based on the nature of the processing into routine calculation processing and condition judgement processing, enabling the processing by the routine processing unit to be performed at high speed. Since the header analysis unit indicates the start of a variable length decode for blocks to the data conversion unit at a predetermined point during the analysis, the routine processing unit can commence its processing sooner. This reduces the total time taken by the decode processing (the combined processing of the sequential processing unit and the routine processing unit) for one macroblock.
Here, the start indication unit may issue the start indication before the analysis unit calculates the motion vectors, and the notification unit may notify the synthesis unit of the calculated motion vectors.
With the stated construction, the start indication unit indicates a start of a variable length decode before the calculation of motion vectors by the analysis unit. This reduces the overall time taken to decode each macroblock by the time taken to calculate the motion vectors.
Here, the control unit may include: an interrupt detection unit for detecting an occurrence of an interrupt that causes a switching of thread; an error processing unit for having the routine processing unit discard data relating to a predetermined block when an error interrupt has been detected by the interrupt detection unit and for changing a state of the header analysis unit to a ready state; and a state management unit for managing a state of the header analysis unit and a state of the audio decode unit based on a type of detected interrupt, each state being one of an execute state, a wait state, and a ready state; and a switching unit for switching a thread assigned to the processor to the header analysis thread when the state of the header analysis unit has changed from the wait state to the ready state.
With the stated construction, the control unit includes an error processing unit in addition to the interrupt detection unit, state management unit, and switching unit. When an error interrupt has been detected, the error processing unit resets every component in the routine processing unit and sets the header analysis unit into the ready state. As a result, the header analysis thread is assigned to the processor and header analysis is performed for a macroblock in the next slice. This means that when an error has occurred, the control unit in the present media processing apparatus can smoothly assign the header analysis thread to the processor, which improves the overall processing efficiency.
Here, the predetermined block may be a macroblock included in the compressed video data in the data stream, the interrupt detecting unit may be capable of detecting a type 1 interrupt, a type 2 interrupt, a type 3 interrupt and a type 4 interrupt, the type 1 interrupt being outputted by the header analysis unit on completing the header analysis process for one macroblock, the type 2 interrupt being outputted by the routine processing unit on completing the decode process for one macroblock, the type 3 interrupt being outputted by the data conversion unit when an error has occurred during a variable length decode, and the type 4 interrupt being outputted by the synthesizing unit when an error has occurred during the motion compensation process, wherein the state updating unit may change the state of the header analysis unit in the state table to the wait state when the interrupt detecting unit has detected the type 1 interrupt and change the state of the state of the header analysis unit in the state table to the ready state when the interrupt detecting unit has detected the type 2 interrupt, and wherein when one of the type 3 interrupt and the type 4 interrupt has been detected by the interrupt detection unit, the error processing unit may have the routine processing unit discard data relating to a predetermined block and the state updating unit may change the state of the header analysis unit to the ready state.
With the stated construction, when an error has occurred during the variable length decode by the data conversion unit or during the motion compensation process by the synthesizing unit, the error processing unit resets each component of the routine processing unit and then sets the header analysis unit into the ready state. As a result, the header analysis thread is assigned to the processor and the header analysis process is performed for a macroblock in the next slice. This means that the control unit of the present media processing apparatus is able to smoothly assign the header analysis thread to the processor even when an error has occurred, thereby raising the processing efficiency.
Here, the interrupt detecting unit may be capable of detecting eight types of interrupts numbered type 1 to type 8, a type 1 interrupt being outputted by the header analysis unit on completing the header analysis process for a predetermined block, a type 2 interrupt being outputted by the routine processing unit on completing the decode process for a predetermined block, a type 3 interrupt being outputted by the header analysis unit on completing the header analysis process for every predetermined block in a frame, a type 4 interrupt being outputted by the data conversion unit if the variable length decode is not complete for the header information when the obtaining unit issues the indication, a type 5 interrupt being a vertical synch signal inputted from outside the media processing apparatus, a type 6 interrupt being outputted by the data conversion unit after the type 4 interrupt when the variable length decode is complete for the header information, a type 7 interrupt being outputted by the memory when the memory does not have data that has been requested by the audio decode unit, and a type 8 interrupt being outputted by the memory after the type 7 interrupt when the data that has been requested by the audio decode unit has been inputted into the memory, the state updating unit changing the state of the header analysis unit in the state table to the wait state when the interrupt detection unit has detected any of the type 1 interrupt, the type 3 interrupt, and the type 4 interrupt, the state of the header analysis unit in the state table to the ready state when the interrupt detection unit has detected any of the type 2 interrupt, the type 5 interrupt and the type 6 interrupt, the state of the audio decode unit in the state table to the wait state when the interrupt detection unit has detected the type 7 interrupt, and the state of the audio decode unit in the state table to the ready state when the interrupt detection unit has detected the type 8 interrupt.
With the stated construction, the state updating unit updates the states of the header analysis unit and the audio decode unit in accordance with the detection of type 1 to type 8 interrupts by the interrupt detection unit. As a result of this updating, the switching unit has the header analysis thread and the audio decode thread assigned to the processor with priority being given to the header analysis thread. The control unit is able to smoothly assign the header analysis thread and the audio decode thread to the processor even when the various types of interrupts occur, so that the overall processing efficiency is raised.
The stated object of the present invention can also be achieved by a computer-readable storage medium storing a program which has a computer decode a data stream including compressed audio data and compressed video data to obtain video data and audio data, the program including: a header analysis process for analyzing a header of a predetermined block included in the compressed video data in the data stream; an audio decode process for decoding the compressed audio data in the data stream; a control process for switching between the header analysis processing and the audio decode process, giving priority to the header analysis processing; and a routine process for performing a decode process for the compressed video data in the data stream, the decode process not including the header analysis process.
Here, the computer may be a media processing apparatus that has a first processor and a second processor, the first processor executing the header analysis process, the audio decode process and the control process and the second processor executing the routine process.
With the stated construction, the first processor prioritizes the execution of the header analysis process and so executes the header analysis process without delay. When execution of the header analysis process is not possible, the first processor executes the audio decode processing, thereby increasing the processing throughput. Since the first processor executes the header analysis processing without delay, the second processor can quickly receive the processing results of the header analysis process and so quickly commences the routine process.
Here, the control process may include: an interrupt detection process for detecting an interrupt that causes a switching between the header analysis process and the audio decode process and a type of interrupt; a state management process for managing a state of the header analysis process and a state of the audio decode process based on the type of interrupt that has been detected, each state being one of an execute state, a wait state, and a ready state; and a switching process for switching, when the state of the header analysis process has changed from the wait state to the ready state, a process executed by the first processor from the audio decode process to the header analysis process.
With the stated construction, the first processor commences the header analysis process as soon as the header analysis process changes to the ready state. This improves the overall processing efficiency.